Variable margin losses for classifier design
نویسندگان
چکیده
The problem of controlling the margin of a classifier is studied. A detailed analytical study is presented on how properties of the classification risk, such as its optimal link and minimum risk functions, are related to the shape of the loss, and its margin enforcing properties. It is shown that for a class of risks, denoted canonical risks, asymptotic Bayes consistency is compatible with simple analytical relationships between these functions. These enable a precise characterization of the loss for a popular class of link functions. It is shown that, when the risk is in canonical form and the link is inverse sigmoidal, the margin properties of the loss are determined by a single parameter. Novel families of Bayes consistent loss functions, of variable margin, are derived. These families are then used to design boosting style algorithms with explicit control of the classification margin. The new algorithms generalize well established approaches, such as LogitBoost. Experimental results show that the proposed variable margin losses outperform the fixed margin counterparts used by existing algorithms. Finally, it is shown that best performance can be achieved by cross-validating the margin parameter.
منابع مشابه
A view of margin losses as regularizers of probability estimates
Regularization is commonly used in classifier design, to assure good generalization. Classical regularization enforces a cost on classifier complexity, by constraining parameters. This is usually combined with a margin loss, which favors large-margin decision rules. A novel and unified view of this architecture is proposed, by showing that margin losses act as regularizers of posterior class pr...
متن کاملRole of Heuristic Methods with variable Lengths In ANFIS Networks Optimum Design and Training
ANFIS systems have been much considered due to their acceptable performance in terms of creation of fuzzy classifier and training. One main challenge in designing an ANFIS system is to achieve an efficient method with high accuracy and appropriate interpreting capability. Undoubtedly, type and location of membership functions and the way an ANFIS network is trained are of considerable effect on...
متن کاملThyroid disorder diagnosis based on Mamdani fuzzy inference system classifier
Introduction: Classification and prediction are two most important applications of statistical methods in the field of medicine. According to this note that the classical classification are provided due to the clinical symptom and do not involve the use of specialized information and knowledge. Therefore, using a classifier that can combine all this information, is necessary. The aim of this s...
متن کاملGeneralization error for multi-class margin classification
In this article, we study rates of convergence of the generalization error of multi-class margin classifiers. In particular, we develop an upper bound theory quantifying the generalization error of various large margin classifiers. The theory permits a treatment of general margin losses, convex or nonconvex, in presence or absence of a dominating class. Three main results are established. First...
متن کاملThe Role of Weight Shrinking in Large Margin Perceptron Learning
We introduce into the classical perceptron algorithm with margin a mechanism that shrinks the current weight vector as a first step of the update. If the shrinking factor is constant the resulting algorithm may be regarded as a margin-error-driven version of NORMA with constant learning rate. In this case we show that the allowed strength of shrinking depends on the value of the maximum margin....
متن کامل